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Abstract: This paper is motivated by the analysis of gene expression sets, especially by finding dif- 
ferentially expressed gene sets between two phenotypes. Gene logj expression levels are highly correlated 
and, very likely, have approximately normal distribution. Therefore, it seems reasonable to use two-sample 
Hotelling's test for such data. We discover some unexpected properties of the test making it different from 
the majority of tests previously used for such data. It appears that the Hotelling 's test does not always reach 
f^ ' maximal power when all marginal distributions are differentially expressed. For highly correlated data its 

O^ ■ maximal power is attained when about a half of marginal distributions are essentially different. For the case 

when the correlation coefficient is greater than 0.5 this test is more powerful if only one marginal distribu- 
tion is shifted, comparing to the case when all marginal distributions are equally shifted. Moreover, when 
the correlation coefficient increases the power of Hotelling 's test increases as well. 






^ ; 1 Introduction 

. ' In many situations statisticians need to test multidimensional hypotheses. In a lot of cases components of 

j^ , observed random vectors are highly dependent, which may change the properties of the tests used. One of 

'^ ' the examples of such data is pr ovided by gene express i on lev els. Gene expressions are highly correlated 



X 



between genes (see for example lA"/efcawov and YakovleM2QQl\i ). Moreover, often the genes are investigated 



not just separately, but also as a set of dependent genes. Therefore one has to deal with multidimensional 

►-j^ , hypotheses and in order to test such hypotheses, gene sets should be expressed differentially. The most 

tH- ' popul ar tests for gene sets are Hotelling's test, N-test and tests derived f rom marginal f-statistics. In the 

CTS . pupers lAckermann and Strimmen (120091) . iGtozto and Emmert-StreiH (120091) . an approach to comparing these 



^^ ' test in various situations was made. Our goal is not to make another comparison, but rather to describe some 

interesting properties of the Hotelling's test which seems to be unexpected. 

o, 

O . 2 Hotelling's test 



One of the most well known tests is f-test. Hotelling's test is an multidimensional extension of f-test. Similar 
to f-test, we can consider both one-sample and two-sample Hotelling's test. One-sample case deals with the 
hypothesis that the expected value of a sample from multidimensional normal distribution is equal to some 
C^ ' given vector. In the two-sample case it deals with the hypothesis of the equality of expected values of two 

samples from multidimensional normal distributions (with the equal covariance structure). In this paper we 
will focus on the two-sample Hotelling's test. 

Suppose we have two independent samples (of sizes n^ and n,,, respectively) from two n-dimensional 
normal distributions with identical covariance matrices equal to 2. In other words, we consider X[, ..., X„^ as 
i.i.d random vectors having N„{fix, E) and Fi, ..., F„, as i.i.d random vectors having N„{py, E) (Xi and F, are 
independent for all / = 1, ..., n/, j = I, ..., ny). For simplicity we assume that n < n^ -\- ny - 1. Our goal is to 
test the hypothesis H : fi^ = fiy against alternative A : fx^ ^ l-iy For this we use Hotelling's test based on the 
statistic 

T^ ^ «x«y (x_Yfs-^(X-Y), (1) 



where X ^ ^^ YZi ^^ ? = i 2,=i ^i ^nd S 
distribution by 

rir + ny — n- 1 






-T ~ F(n, n^ + rty - n - 1). 



T is related to the F- 



(2) 



n(n^ + n^ - 2) 

For more details about Hotelling's test see, for example. lC/zaffe/J and Co//wj (Il980r) . We made the assump- 
tion n < Hf + My - 1 for two reasons. For n > rijc + riy- 1 the estimate 5 of 2 results in an irregular matrix, so 
that 5 ' does not exist and moreover numerator of (O is non-positive as well as the degree of freedom of the 
F-distribution. In such situations it is possible to use some pseudo-inversion of S and in order to estimate 
/?-value of H, we can use permutations of (Xi , ...,X„^, Yi, ..., F,,, ). 



3 Hotelling's test for strongly dependent data 

As it was mentioned above, genes are highly dependent and we will suppose that their lo si expression level s 
have approximately normal distributions. Many papers work with gene sets (for example lffgrrv et all (120081) ^ 



instead of genes alone and therefore deal with multidimensional hypotheses. It seems to be reasonable to 
use Hotelling's test in this situation. 

Assume that we have two multidimensional samples and need to test the hypothesis suggesting the 
equality of expected values in these two samples. Assume for simplicity that all elements on the main 
diagonal of the covariance matrix 2 for both samples are equal to 1 and all other elements are equal to p > 0, 
i.e. 

{ I p p ... p '^ 
p I p ... p 



Z = 



V P 



p 1 



Further on, we assume that //^ = (0, ..., 0)^, but fiy has first m elements equal to 1 and the others equal to 
0, i.e. 

fiy^{l,...,l,0,...,0f. 



in II— m 



For large n^ and ny the matrix Z and its estimate S are approximately the same as well as the differences 
between the expected values (p.^ - Py) and between the mean values {X - Y). When dialing with real data, 
Hf and n, might not be large enough, but for theoretical reasons we may use the approximations S ~ I, and 
X - Y ^ Px - Py. In this case 5"' =s Z"', that is 



z-' = 



where a = ^,%';l::l^%-, and^ = (i-p)(U(«-i)p) - 



a 
-P 

-P 



-P -P 
a -p 



... -p 
... -p 

-P a 



For fixed n v and n,, we can consider the fraction 



^kof 



Hotelling's statistic ([TJ as a normalizing constant. Let us denote T*"^ Hotelling's statistic with Z ' instead of 
5 ' and p^ - /i, instead ofX-Y divided by the constant k. Therefore, we have 



T^lk. 



r^ = (p,-Pyf^-\p_,-p,:) 



= (l,...,l,0, 



= ma - (m — m)/3 = 





fa -p -p 




o,...,o) 


-f3 a -p 




n-m 


I -P 


-P 


m(l + (n - 2)p) - m(m - 


Dp 





( ^ ] 


-P\ 
-P 


1 



a ) 


, , 



m(l + (n — m — l)p) 



(3) 



(l-p)(l+(«-l)p) (l-p)(l+(«-l)p) 

Let us note that it does not matter if//,, consists of ones and zeros or equals to a constant a and zeros. In the 
latter case, statistic T*^ would be multiplied by a^. Now we will work with statistic T*^ and investigate its 
behavior 

If we changed m to m + 1 (meaning that we add one more different marginal distribution) we would 
expect that the statistic T*^ increases and that so does the power of Hotelling's test. We need to check if it is 
indeed the case. For better understanding let the number of ones in ju, be the index of T*^ (we will write it 
only when it is needed). Now we change m to m + 1 = h and we have 

- y „, +a 



'- m+l 



2m/3. 



If we expected that T*^ is an increasing function of m then a - m^p should be greater then zero. But we have 

1 + (« - 2)p 2mp 1 + (n - 2m — 2)p 



a — 2mp — 



(l-p)(l+(n-l)p) (l-p)(l+(«-l)p) (l-p)(l+(«-l)p)- 



Since the denominator is greater than zero, then a - Imp > only if 2mli-n ~ WTi ^ fi- 



ll means that for 



not very small values of p's and m > | - 1 the statistic T*^ is a decreasing function of m. This means that 
maximal power of Hotelling's test (as a function of m) is not always attained for m = n but for p's which are 
not very small we have maximal power for m near |. Some examples of the behavior of T*^ as a function of 
m are illustrated on figure [1] 

However, this issue is not the only one that is surprising about Hotelling's test. Now we look if T^^ is always 
lower than T*^. It is the case when one different marginal distribution influences more than all n different 
distributions. So we need to compare a with na - n{n - \)p. We have 



y,,2 _ j*i ^ Q, _ „Q, + „(„ _ j)y3 = (« - 1) 



(l-2p) 



(l-p)(l+(«-l)p) 



So T*^ - T*^ < only ifp < 0.5. Therefore we can say that forp > 0.5 Hotelling's test has better power for 
alternative with only one marginal shift than for alternative that all marginal distributions are equally shifted. 
It can be seen from figure[T]as well. Moreover, the statistic T*^ is an increasing function of p, that may seem 
surprising as well. 

4 Hotelling's test for two-dimensional data 

Let us look at Hotelling's test in the two-dimensional case. As in the previous case, we will consider the 
two-sample problem, but now we will generalize the difference of expected values of these two samples. 
Suppose that yu., - yUy = (oi , 02) and that the covariance matrix is 



2: = 



1 P 
P 1 



n= 10 ; rho=0.1 n=10;rho=0.3 n=10;rho=0.5 n=10;rho=0.7 n= 10 ; rho= 0.£ 
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n= 15 ; rho= 0.1 n= 15 ; rho= 0.3 n= 15 ; rho= 0.5 n= 15 ; rho= 0.7 n= 15 ; rho= OS 
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n= 25 ; rho= 0.1 n= 25 ; rho= 0.3 n= 25 ; rho= 0.5 n= 25 ; rho= 0.7 n= 25 ; rho= 0.£ 
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n= 40 ; rho= 0.1 n= 40 ; rho= 0.3 n= 40 ; rho= 0.5 n= 40 ; rho= 0.7 n= 40 ; rho= 0.9 
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Figure 1: Plots ofT*^ for« = 10, 15, 25, 40;p = 0.1, 0.3, 0.5, 0.7, 0.9; andm = 1, ...,«. Notice: each plot is 
differently scaled! 



Then inverse of S is the matrix with diagonal elements a 



1 



-p 



(i-p)(i+P) 



and off-diagonal elements -/3 



(i-p)(i+p) 



Then 



T* = aflj + aflj - 2y6flifl2- 

First we consider that ai = 1 and 02 - 0. Then T*^ = a. Now we will investigate for which ai,a2 e R 
statistic T*^ = a. That is, we need to solve an equation 



cflj + aa2 — 2ySaia2 = a. 
After dividing both sides of equation (HJi by a we get 

flj + flj ~ 2pa\a2 —1=0. 
For fixed a 1 equation (|5]l is quadratic in 02 with the roots 



(4) 



(5) 



«2,2 = 



2pfli ± J(2pai)2-4(fl2_ 1) 



It is defined only if (2pfli)^ -4(flj- 1) > 0, i.e. for|fli| < Jj^- Some plots of the solutions of the equation 

Q for different values of the correlation coefficient p are given on figure |2l We can see that the plots of 

these solutions produce elliptic curves. Let us rotate these ellipses by the angle ip = 11/4 clockwise. To do 

this, we use transformation 

V2 V2 
ai - xcos(f — ysmcp - —r-x — y, 

V2 V2 
02 — X sm (fi + y cos (p — x H y, 

where x and y are new rotated coordinates. After substitution into ^ it gives 

^ V2 V2 . ^ V2 V2 . ^ ^ V2 V2 ^^ V2 ^/2 ^ 
(—-—y) +(—-+—3') -2p(—x-—y){—x+—y) 

2 2 

= x2(l-p)+/(l+p) = ^ + J^ = l, 

where a — Jjr- and b - Jj^ are respectively the major radius and the minor radius of the elhpse. Since 
a > b, the Hotelling's test has the weakest power in the direction of oi = a2, while the fastest increase of 
its power is observed towards the direction of ai - -ai- For example, forp - 0.9 we have a - 3.162 and 

b - 0.725. It means that for a\ - 02 - a/ ^3^ - 2.236 Hotelling's test has approximately the same power 

as for a\ - \, 02 - (or for a\ - -02 - -J °'™" — 0.513 as well). So, if there is only one marginal distri- 
bution shifted by one unit, then the power of Hotelling's test is approximately the same as if both marginal 
distribution were equally shifted (in the same direction) by 2.236 units (for the shift in opposite direction it 
should be only 0.513 unit). These results are in contradiction with other multidimensional tests. For exam- 
ple, consider the test based on marginal f-statistics. The power of this test is higher if both distributions are 
shifted by the same amount (both f-statistics are "large", not depending on direction of shift) than if there 
was only one marginal distribution shifted (one f-statistic is "near" zero). 

5 Theory and reality 

The analytical results obtained above should be verified by checking if actual Hotelling's test outcomes 
correspond to the analytical results regarding real data. In this section we will compare the behavior of 
theoretical Hotelling's statistic T*"^ with real Hotelling's statistic T^. For large n^ and n, we assumed that 
T*^ ~ T^ Ik, where k - ^7^. Constant k changes as n, and n,, change. It is reasonable to divide Hotelling's 

statistic T^ by k instead of multiplying T*^ by k in order to be able to compare how do T^ and T*~ differ for 
various n^ and n,,. 

In order to compare the actual results with the analytical ones, we did the following simulations. All data 
were simulated from n-dimensional normal distributions. Consider three different values for the number of 
genes in a gene set. We take n = 10, « = 15 and n = 25. All simulations were performed for three different 
values of the correlation coefficient p : p — 0.1, p = 0.5 andp - 0.9. In order to compare the behavior of 
Hotelling's test for various sizes of samples we took three choices of n.t and n^: n^ - n^. - n, n^ - n^ - \An 
and n J = «>• - '2. An. The value m which is the number of false marginal distributions varies from one to 
n. The shift value for each of the different marginal distributions is set to one. The theoretical Hotelling's 
statistic is calculated according to (O. Real Hotelling's statistic is estimated from 1000 simulations for each 
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Figure 2: Plots of solutions of equation (??) for two-dimensional case for rho - 0.25;0.5;0.9. Notice: each 
plot is differently scaled! 

case (as the mean of T^ jk obtained from the simulations). 

Plots of our simulated cases are shown on figure[3] We can see that for all simulated situations, the shapes 
of real and theoretical Hotelling's statistics are similar. The only difference is in the heights of these curves. 
For small n , and n,. statistic T^ has higher values than for large n^ and iiy. The reason for that stems from the 
inaccurate estimates of the expected values and of the covariance matrix. However, we observe that with the 
increase of n., and n,., statistic T^/A: goes to T*^ relatively fast. Therefore, the behavior of Hotelling's test for 
real data is expected to be very similar to the behavior of statistic T*^. 

In previous section we saw that for the two-dimensional case the plotted shifts with equal values of the 
power of theoretical Hotelling's test form elliptic curves. Hotelling's statistics T^ are random variables. 
Therefore, we can only estimate if their expected values form elliptic curves when plotted. To check this 
we did following simulations. Instead of calculating the shifts for which Hotelling's test has equal powers, 
we took the points provided by the elliptic curves observed for theoretical Hotelling's statistics. For each 
pair of these points (01,02) we did 1000 simulations and calculated Hotelling's statistic. We estimated 
the expected value ET^/k as the mean for these 1000 repetitions. We divided Hotelling's statistics by k 
for better understanding how fast these statistics go to T*^. We did this simulation for the values of the 
correlation coefficient p - 0.3 and p = 0.9 and as the number of observations in each sample we took 
Hx = ny = 5, n.t = n,. = 10 and n^ - riy - 20. Results of our simulation are given in Table 1. We observe that 
estimated mean values of T^jk are not very different, that they go to T*^ and that their variance decreases 
with increasing number of observations. Clearly, these points form elliptic curves. Hence, we can claim that 
the real Hotelling's test behaves very similar to the theoretical one and the theory derived for the theoretical 
test holds for the real Hotelling's test as well. 



6 Discussion 

In this paper we have discovered that two-sample Hotelling's test (for testing the equality of the expected 
values of two samples from multidimensional normal distribution with equal covariance structure) has some 
unexpected properties. At first sight, one could expect that with a larger number of false marginal distribu- 
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Figure 3: Comparisons of theoretical statistics T*^ and real Hotelling's statistic T^/k for number of genes 
n = 10 15, 25 (from the top to the bottom); for correlation coefficientp - 0.1, 0.5, 0.9 (from the left to the 
right) and number of observations in each sample n^ - n^. - n (denoted by ' +'), Ux - n^. - I An (denoted by 
'x') and Ux - tiy - 2 An (denoted by '•'). The theoretical statistic T*^ is denoted by 'o'. Number of different 
marginal distribution m is set from one to n. Notice: each plot is differently scaled! 



tions the power of this test increases. But we have discovered that this is not true in general. For highly 
correlated and high dimensional data (such as data sets of gene expressions) maximal power of Hotelling's 
test is reached when only about one half of the marginal distributions are shifted. We have found out that 
when the correlation inside the sample is greater than 0.5, then the Hotelling's test can have a better power 
if only one marginal distribution is different, as opposed to the case when all marginal hypotheses are false. 
Moreover, the power of Hotelling's test increases for higher correlations. That observation may seem some- 
what unexpected as well. We have investigated Hotelling's test in detail in two-dimensional case. We have 
found that properties of this test are much different from ones of the tests based on marginal f-statistic. All 
reasonable tests based on marginal f-statistic do not depend on the direction of the shift. But the power of 
Hotelling's test increases very slowly if both of the marginal distributions are equally shifted and increases 
much faster if marginal distributions are shifted in opposite directions. Moreover, alternatives with equal 
values of the power form ellipsoids. 



Table 1: Results of simulations of two-dimensional adjusted Hotelling's statistics T Ik with n, — n^ — viy 
observations for each sample and correlation coefficient p. T*^ stands for theoretical Hotelling's statistics 
and (fli, fl2) is difference between expected values jj.^ - ij.y of these samples. On bottom line is the estimate 
of variance of each column. 
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